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I. INTRODUCTION 



A. Objectives of the Conference 

A conference entitled ”The Relationship of Automatic Data Processing 
Training Curriculum and Methodology in the Federal Government” was con- 
ducted in Washington, D.C. during the period May 15-20, 1967 by the 
Association for Educational Data Systems and sponsored by the United 
States Office of Education. This conference had as its objective the 
determination of recommendations for the establishment of an effective 
and efficient Automatic Data Processing (ADP) training program utilizing 
new instructional methodologies. More broadly described, the purposes 
were: first. Federal Government agencies such as Office of Education, 
Civil Service Commission, National Bureau of Standards and the Bureau 
of the Budget have an immediate and pressing need to define those 
currIcCilums or bodies of subject matter knowledge required by two groups 
of federal employees: (a) the computer systems analysts and (b) the 
manager users. The participants at the conference attempted to determine 
the behavioral objectives or train! ng/development goals for these two 
groups and to list in general terms the subject matters which should 
be contained in training programs for them. The second, and equally 
important purpose of the conference, was to determine to what extent 
educational multi -media courses could be utilized to efficiently 
produce effective training results in filling the enormous need for 
trained systems analysts and informed managers at all levels. 

The major portion of this paper is directed to reporting the re- 
sults of the conference in achieving the foregoing objectives. However, 
the major impact of the conference and this report is not contained 
herein but will come from the actions taken to implement the recommenda- 
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tions resulting from these meetings. The second stage of implementing 
ADP training with multi -media courses will come when, and if, a pilot 
project is established to implement the recommendations contained in 
this report. Moreover, It is hoped that it will be the intention of 
the Federal Government to achieve from the pilot project an operable 
prototype training program; rather than an experiment. 

Finally, there was an expression of hope that the recommendations 
of the conference and the pilot project to follow would result in the 
achievement of these additional objectives: 

1. Development of a methodology to determine who needs training 
in ADP 

2. Development of a sequential and modular array of subject 
matter curriculum segments which would permit a training 
approach that exposes the student only to those modules which 
he requires. This would avoid the duplication and overlap so 
prevalent in existing programs 

3. ^velopment of a technique whereby the practitioner or student 
could diagnose his needs for training. 

B. Events Leading to the Conference 

In order to better understand the results and Impact of the 
conference, it is helpful to briefly review the historical developments 
leading to its implementation and to this resulting report. 

The need for new and innovative training programs has been under 
discussion for several years. Preliminary discussion began in 1964 
when the United States Civil Service Commission released a report 
through the House Post Office and Civil Service Committee titled "A 
Study of the Impact of Automation on Federal Employees,” Page 37 of 
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the report (under Section V, Summary of Findings and Statements of 
approved actions) contains a reference to the establishment of a Joint 
Agency Commission. 

3. Joint agency-CommIsslon action Is being taken to prepare 
for the future. In addition to improved forecasting of 
changes: The Civil Service Commission Is establishing 

machinery for continuous discussion and action with the 
agencies In all personnel management program areas affected 
by automation. Cognizance will be maintained of develop- 
ments In automation technology and applications. 

Future problems will be Identified and action taken to 
mitigate their effects, as early and as rapidly as possible. 

The Implementation of this recommendation was fulfilled In a 
letter of October 29, 1964 from Mr. Irving Kator, Executive Vice 
Chairman of the United States Civil Service Commission. In this 
letter, reference was made to the establishment of an Interagency 
Advisory Group,, Committee on Automation and Manpower (IAG-204). Mr. 
Hugh W. Scott, Personnel Management Special 1st, Bureau of Programs 
and Standards was named Chairman of the (Committee. 

The first meeting of lAG-204 was on December 16, 1964. As stated 

in this meeting the purpose of this Committee was: 

’’to provide a medium for executive branch collaboration 
on the personnel management and manpower aspects of 
automation In order to facilitate Its utilization and 
to mitigate Its adverse effects on personnel. ADP will 
be the primary concern of the committee, but the effects 
of other types of automation will also be of concern to 
It." 

Five task forces of IAG-204 were then established In March of 
1965. It was the defined responsibility of these task forces to ex- 
amine their assigned subject area in depth and to make appropriate 
recommendations. These task forces were designated as follows: 

Task Force 1 - The need for a special Civil Service examining 
program for computer specialists. (Mr. Sam McDonald - 
Census Bureau - Chairman) 
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Task Force 1 1 - Increased education of managers and key per- 
sonnel in how the computer functions, its potential, and 
its shortcomings. (Mr. Fred Dyer - Department of the 
Navy - Chairman) 

Task Force III - Improving placement programs for displaced 
workers, (Mr. Chet Evans - Department of the Treasury - 
Chairman) 

Task Force IV - Establishment of (5overnment-wide standards 
for projecting and reporting direct and Indirect effects of 
automation on manpower requirements. (Mr. Leon Greenberg - 
Department of Labor - Chairman) 

ITask Force V - Keeping abreast of advances in automation 
technology and applications, and evaluating the extent 
and time of impact of the advances on personnel and man- 
power requirements, (Mr. Howard Gammon - National Bureau 
of Standards - Chairman) 

At approximately the same period in time (March 1965), an ex- 
tremely important and significant document was also released, making 
general reference to personnel training. This is commonly referred 
to as Senate Document 15, carrying the full and complete title of 
"Report to the President on the Management of Automatic Data Processing 
in the Federal Government." The report was prepared by the Bureau of 
the Budget and submitted through the Senate Committee on Government 
Operations. Page 3 of this report refers to the personnel problems 
facing the Federal Government and makes specific reference to the 
Civil Service Commission Study' (already mentioned). 

This report led to Circular No. A-71, from the Executive Office 
of the President, Bureau of the Budget and dated March 6, 1965. 

Circular A-71 defined the responsibilities for the administration and 
management of automatic data processing activities. Responsibilities 
for personnel training were assigned to the Civil Service Commission. 
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6. Responsibilities of the Civil Service Commission . — The 
Civil Service Commission Is responsible for providing ex- 
ecutive branch-wide leadership and assistance In the personnel 
rr.anagement and manpower aspects of automatic data processing. 
In this connection, the Commission will foster programs 
designed to — 

(a) Staff automatic data processing actIvlTles effectively 
by, among other things, (1) formulating position classifi- 
cation and qua inif feat ion standards, (2) developing necessary 
special recruiting techniques, (3) devising improved testing 
and selection devices and (4) stltfiulatlng and coordinating 
necessary training. 

(b) Educate executives and other key personnel to achieve 
greater effectiveness In ADP management. 

(c) Anticipate and minimize, to the greatest practicable 
extent, any adverse effects of automatic data processing 
upon the people Involved. 

(d) Provide- a medium within the executive branch to focus 
and coordinate preparation for the future personnel manage- 
ment and manpower effects and requirements of automatic data 
processing. 

These documents further indicated the importance of the work 
already begun by IAG-204, primarily through the previously mentioned 
task forces. 

The organization of the task forces under IAG-204 called for 
ifask Force V to establish sub-groyps for more detailed examination of 
identified areas of automation advances. One of these sub-groups was 
the Subcommittee on Programmed Instruction, chaired by Dr. Chester L. 
Guthrie, National Archieves and Record Service. The purpose of the 
Subcommittee was to Investigate the status of CAI (Computer Assisted 
Instruction) and Its potential effects on the training of Federal 
personnel In ADP. After conversations with Mr. Francis Keppel, 

Dr. R. Louis Bright, Mr. David Bushnell, Colonei Gabriel 0»FIesh and 
others, the Subcommittee generally agreed that programmed instruction, 
in paf'^Icular computer assisted Instruction, h^l-d great promise as a 
possible means of solving certain personnel training problems in ADP, 
and should be explored in greater depth. 
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Thls agreement was transmitted in the form of a recommendation 
to the full Committee on Automation and Manpower. This recommendation 
was subsequently sent to the Civil Service Commission by Mr. Scott, 
Chairman of IAG-204. After several internal communications, the Civil 
Service Commissioners (Mr. John Macy, Chairman; Mr. L. J. Andolsek 
and Mr. Robert Hampton) authorized proceeding with the concept of CAl 
for ADP training and assigned CSC responsibility to the Office of 
Career Development. 

Discussions for funding a program involving such training concepts 
were then directed to the United States Office of Education, where 
plans for a proposed working conference were discussed, it was deter- 
mined that this could possibly result In an RFP for a CAl demonstration 
project for training Federal employees in ADP. 

C. Pai"tlc-i.pants 

An outstanding group of conference participants representing 
experts in subjects related to the content of the conference were 
assembled. These included: 

1. Fifteen topical specialists in subject matter, curriculum 
and education, and educational media technology. 

2. Ten Federal (Sovernment officials concerned with the admin- 
istration of ADP programs and associated training. 

3. Two major resource specialists in the field of programmed 
instruction and CAi. 

4. Eighteen additional resource specialists, including: 

a. Top Federal Government ADP Executives 

b. Educational Technology Manufacturers 
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c. ADP training consulting firms 

d. ADP executives of Industrial firms. 

A list of participants Is contained In Appendix II. 

This report would not be complete without calling attention to 

* 

those Federal Government officials whose efforts were particularly sig- 
nificant In the events leading up to the conference and In their leader- 
ship during Its deliberations. These are: 

Mr. Howard Gammon National Bureau of Standards 



Dr. Chester Guthrie 



Mr. Joseph Lowell 
Dr. Richard Otte 



National Archives 

Civi I Service Commission 



Office of Education 



Civil Service Commission 



Mr. Hugh Scott 
D. Why ADP Training 

Recent reports by the Civil Service Commission, Including the 
Presidential Task Force on Career Advancement, estimate that two and 
one half million federal employees need some kind of training In either 
new skills or in updating the skill areas in which they now work. Why 
then, should computer training be chosen as the subject of this con- 
ference? The answer becomes rather clear; first, that ADP skills are 
among the most critical in both the government and private sectors. 

Since these skills are fundamentally the same in both sectors, training 
improvements developed by the Federal Government will have almost total 
transferability to industry. A second reason for choosing this area 
of ADP training and development is the conclusion that the innovative 
methodologies developed for this type training can be utilized across 
a wide range of other subject matter and skill areas. In summary, the 
results of this conference could stimulate thinking and action that 
might have a major impact on ail types of training, not just In automatic 
data processing. 
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II. THE ENVIRONMENT AND NEED FOR 
ADP TRAINING AND DEVELOPMENT 



A. Impact of the Computer 

It is probably true that no event since the Industrial revolution 
will have as profound an impact on technology as the computer. Indeed, 
as Mr. John Eberhart of the National Bureau of Standards expressed It; 
”We are In the process of passing from an Industrial to a Systems 
Revolution re: the impact of technology on society." The Implications 
for Government and industry are essentially the same although the 
applications are somewhat different. Automation as found in industry 
for production planning, process control, and the like are applied in 
the Federal Government to large information processing and communi- 
cations networks in which computers are an essential segment. 

The impact of the computer in the Federal Government has been 
described: 

No single technological advance in recent years has contributed 
more to effectiveness and efficiency in Government operations 
than the development of electronic data processing equipment. 

Most of the important advances that have been made in such 
diverse fields as space exploration, research of all types, 
weather forecasting, and atomic energy would not have been 
possible without the computer. In the field of large-scale 
clerical operations such as Insurance processing, checkwrltihg, 
and the tax system, the computer has materially assisted in 
producing major economies. Furthermore the computer is be- 
coming Increasingly useful to managers In solving complex 
problems Involving interrelated types of Information. The 
most notable of these have been In the military areas and in 

supply management the Impressive advantages to the 

Government already achieved through automatic data processing 
(ADP) are but steppingstones to the future. 



^U. S. Senate, Committee on Government Operations. The Management of 
Automatic Data Processing in the Federal Government . 89th Congress, 
1st Session, March 4, 1965. 
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in industry, advances are equally spectacular. In the automobile 
Industry for example, given the existing proliferation of products, it 
is estimated that it would take all existing employees to process the 
necessary paperwork under former manual systems methods. 

The extent to which better applications and Improvements can be 
made with computers appears to be limited only by the availability of 
trained people. This training problem and Its implications have been 
highlighted by a Presidential Task Force on Career Advancement: 



Without question, the single most critical problem In ADP 
training is the need for understanding and support by top 
management 



and 

The second most important problem in ADP training Is the 
acute shortage of ADP personnel.^ 

B. Growth of Computers and Computer AppI i cations 

It is difficult to trace historically or perdlct the growth of 
computers because of rapidly changing technology. What is the yard- 
stick of measurement? To say that installations have grown from 1,000 
in 1955 to an estimated 50,000 in 1970 (AFIPS estimate)^ is meaningless 
unless consideration is taken of some measure of the fantastic Incrase 
in capacity (e.g., 1000 to 1 reduction in cost to perform calculation). 
However, by any measure the actual and predicted growth is phenomenal. 



^National Bureau of Standards, Training for Automation and Information 
Processing in the Federal Service. October , 1966. 



^White House; President’s Science Advisory Committee, Computers in 
Higher Education, The White House,: February, 1967, p. 58. 
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Accordlng to the AFIPS study, the actual (1965) and estimated 
(1970) installations and the impact on manpower needs is summarized: 





1965 


1970 


Total Instal 1 at Ions 


30,000 


50,000 


Systems Analysts 


60,000 


200,000 


Programmers 


60,000 


200,000-650,000 


Operators 


43,000 


80,000 


The impact of computers 


i n the Federa 1 


Government is difficult 


to assess due to the large amount of effort 


utilized on government 


work (contractors, etc.) outside the government. Expenditures on hard- 



ware and software alone during the three most recent fiscal years 
averaged $840 million per year within the Federal Government. Personnel 
implications are clear when account is taken of the growing share of 
the expenditures devoted to software (42 percent in FY 1964 to 51 per- 
cent In FY 1966). 



No precise measure of the impact of computers in industry Is im- 
mediately available. However, It Is expected that relative to government 
use, computer usage In Industry will expand more rapidly because of the 
comparative few in that sector and the potential for additional appli- 
cations. 

In education, the President’s Science Advisory Committee expressed 
the situation as follows: ’’After growing wildly for years, the field 
of computing now appears to be approaching Its Infancy.” In 1965 
the capital value of college and university computers was one twenty- 
sixth of the U.S. total and the cost of computers used in Instruction, 
one-two hundred twentieth of this total. Annual expenditures by 1971-72 
will amount to about $400 million 4f the recommended level of usage is 
atta i ned . 

The implications for training In ali sectors is evident. 
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C. The Role of the Federal Government In ADP Training 

Aside from the Federal Government’s vast expenditures in education 
and its interest in training at ail levels, comments of conference par- 
ticipants and other evidence point to additional reasons for a specific 
Interest In ADP training. First is the tremendous need for trained 
personnel within government, present and future. This need will be 
outlined later in this report. Secondly, it Is clear that government 
is in serious competition with industry and other users of computers 
for ADP personnel. Efforts directed to training the universe of ADP 
personnel will relieve the shortage and turnover in government. Finally, 
by developing improved methods and technology for ADP training, the 
government will have partially solved the problem of education and 
training in many other subject areas. 

p. The Computer Systems Analyst (S/A) in the Federal Government 

As background for placing ADP training in perspective, it was de- 
sirable at the conference to develop Information concerning the tasks 
and skills of a Systems Analyst (S/A) and some definition of the need, 

the input sources, and a general measure of training needs. This back- 
ground is reported here. 

The Systems Analyst-Responsibilities and Duties 
The Civil Service Commission has recently established a new 
position description for Computer Systems Analyst (GS9-12). This was 
taken as the starting point of the conference and Is quoted here; 

"Computer Systems Analysts develop basic plans or "computer 
applications" by which subject-matter processes can be organized 
and accomplished by computer methods. They require a compre- 
hensive understanding and analysis of subject-matter work 
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processes, actions, criteria, as well as supporting controls, 
reports, documentation, etc.. Involved In the function to be 
automated. Also essential is the ability to devise procedures, 
to develop methods for generating and processing data, and to 
Integrate these into data processing systems and plans. 

Typically, specific assignments may Include: Feasibility or 
"profit-ability” studies; the development of detailed systems 
logic charts and diagrams; the development of data reduction 
and coding Instructions, dictionaries, data banks, and the like. 

In addition, these positions require a substantial knowledge 
of computer capabilities and processes, and a basic understand- 
ing of programming principles and methods. They analyze and 
organize subject-matter work processes and functions so that 
they can be converted Into workable computer programs and 
routines. Furthermore, systems analysts must be able to 
foresee some of the specific problems posed In the. subsequent 
programming processes required, as well as some of the possible 
solutions to such problems.” 

This definition Is in terms of output — what Is the end product — 
a typical limitation of job description. The InformatIon.:needed to 
develop curricula Is what the systems analyst puts Into the system — 
what he brings or should bring to the system. 

There was general agreement amohg participants that this descrip- 
tion fairly represented the job of a S/A. Some felt, however, that the 
depth of skills Inferred by the position was greater than the supply 
of personnel which could be hired or retained at that grade level. 
Reflecting this feeling, one participant commented "this description 
Is for a much heavier person. ..consequently, persons doing systems work 
are frequently occupying other positions and higher grades.” 

A slightly different view of the S/A was gained from top Federal 
ADP executives and consulting firms engaged In systems training. Their 
summary of the responsibilities leaned more to the "logical problem 
solver and designer” approach as expressed In these typical comments 
regarding what was expected of an analyst: 
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”An understanding of the Intellectual tools capable of being used 
to look at the relationships between complex activities." 

"Probe and evaluate objectives. Determine best methods for achieve 
ment. Design systems to accomplish and follow up to assure con- 
formance." 

"Find the simplest ways of taking each objective In implementing 
a program and devising the methods and procedures that would 
attain the end result forecast." 

"Plan, factfind, analyze, determine findings, develop general 
systems design and documentation and oversee implementation." 

"Design of workable problem solutions for users." 

"Plan and design information systems In cooperation with various 
user groups in an organization. Deeply involved In implementa- 
tion." 

"To take a logical approach to improvement of the system of 
the organization." 

For purposes of ADP training, perhaps the most significant state- 
ment was the one developed at the end of the conference. This defined 
the objective of the training program as: 



"To provide those Individuals who have adequate background 
in computer programming with the knowledge and skills necessary 
to contribute meaningfully to all basic phases of a systems 
project under competent technical supervision." 

None of the foregoing views of the job or responsibilities of the 

systems analyst are In conflict. Indeed, all were taken into account 

in developing the curriculum of the training program. 



The Need for Systems Analysts 

The current and future shortage of systems analysts had previously 
been adequately documented and was reinforced at the conference. The 
1966 Presidential Task Force on Career Advancement labeled the acute 
shortage of ADP personnel second in criticality only to the need for 
understanding and support by top management. This shortage was 
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attrlbu+ed to the Increasing numbers of Installations and applications 
as well as the recruiting and pay situation In Industry as compared to 
government. Turnover was determined to be over 18^. This same report 
estimated the S/A shortage In the U.S. as 35,000 and the 1970 training 
needs as 130,000. 

The on-board count of S/As In the Federal Government and the 
future need Is difficult to determine due to the fact that: (a) S/A 
duties are being performed by other personnel, (b) discrepancies between 
military vs. cl vl I Ian job descriptions, (c) the large numbers of S/As 
working for contractors not accounted for Individually, and (d) the 
high number of S/As being shared by agencies of the government. There 
Is unquestionably a serious shortage of S/As. One medium size agency 
stated that 45 vacancies for S/As now exist out of a total of 181 
authorized positions. 

Quantitative estimates of the need for S/A training In the future 
are difficult to determine accurately. However, a reasonably good 
estimate of the need did emerge from the conference. 

E. Numbers of Systems Analysts Needing Training (est.) 

li Number In Government 

a. Direct, full involvement and needing full training 

(1) Systems analysts and senior programmers performing 

System analysis activities as a part of their programming 
activities. 18,000 

(2) Management analysts most of whom should be fully 

9,000 
27,000 



trained In system analysis procedures. 
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b. Closely related and needing considerable training 

(1) Subject matter specialists oriented to, and partici- 
pating in, systems analysis 30,000 

(2) Administrative officials In responsible staff positions 

who require some knowledge of system analysis 
procedures 30,000 

60,000 



2. Number in Industry 

a. Available systems analysts (1966) 60,000 

b. Additional needs (1966) 35,000 

95,000 

c. Other related occupations needing considerable 



training in system analysis (1966) 200,000 

Total 295,000 

d. Total needs for analysts by 1970 200,000 

e. Total needs for other related occupations 

by 1970 300,000 



Tot a I both'! needs 500,000 



3. Turnover in Government 

a. From one systems analyst job to another 
(change of agencies) 

b. Leaving Government 

c. Hires from outside Government 

4. New input to "profession" 

a. System analysts and senior programmers 
expected to do systems analysis 
(System analysts per se - 2,000 p.a.) 



\ 6 % 

2 . 2 % 

21 % 



3,000 p.a. 
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5. Extent of need for updating 

a. All system analysts need at least 5 days 
of intensive updating a year. 

b. Existing system analysts needs (other than 
updating) 

(1) Programmer background only - 

6 weeks of general management principles 
and methods 

(2) General management analyst background only- 
4 weeks of basic mathematics 

4 weeks of "hands on" Instruction in a 
basic programming convention 

Sources of Input to Systems Analyst Manpower 
It was important to establish the major sources of input to the 
supply of systems analysts in the Federal Government In order to 
subsequently determine in the conference a recommended subject matter 
training program for each major source of input. Although no quanti- 
tative measure of the size of each source could be determined at the 
conference, there was general consensus that the sources in order of 
quanitity of input were: 

(1) Programmers 

( b ) Coll ege h I res 

(c) Related systems jobs 

Technicians and operators 



Methods analysts 
(4) Subject matter personnel 



F. The Manager in the Federal Government 



The Presidential Task Force on Career Advancement adequately 
stated the need for managerial training at all levels: 



“Vnthout question, the single most critical problem in 
ADP training is the need for understanding and support 
by top management. .. .Although confirming data are not 
available to support the point, it is strongly believed 
that as many as 30 percent of upper level Federal Govern 
ment managers and long service career employees do not 
understand, or even fear, the advent of ADP operations 
in their areas. 

Parallel to the need for top management understanding 
and support is familiarization training for middle 
management and staff personnel.” 



Participants at the conference were unanimous in the view that 
the majority of managers need familiarization with the computer and a 
remarkable degree of unanimity existed v/ith regard to what and how he 
should learn it. This is not surprising in view of the existing and 
projected application of the computer into increasing phases of govern- 
ment operations. Closely associated with the need for the manager to 
know the computer as a user -is the need for him to know it in order to 
close the ”gap” between the ADP person and the manager, a gap that 
unquestionably is costly in terms of tiem, proficiency, and utiliza- 
tion of computer capability. 

Although there was unanimous opinion among the participants that 
manager users contained a vital and large group of potential candidates 
for a training/fami Harization program, it was difficult to estimate 
a precise number. This difficulty was due to three basic character- 
istics of the managerial and executive work forces: 
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(1) Impending retriement of a large percentage of 
Federal Government managers (One estimate was 
that 10 fi In the^rades 11-16 v/ould retire in two 
years). The question arises as to what extent. 
If any, these persons should be trained; 

(2) The extent to which the chief executive of the 
agency understood, appreciated, and would permit 
or encourage agency employees to attend ADP 
familiarization or training programs; 

(3) The extent to which ADP development programs 
should be directed to specific subject matter 
content or peculiar agency uses rather than a 
general overall familiarization program. 



In spits of the difficulty of forecasting the need for managerial 
training and development, the conclusion clearly emerged from the con- 
ference that an enormous backlog did exist. An examination of the 
data below which was a consensus of the thinking of the conference 
experts will provide a measure of the need for managerial training. 

G. Some Estimates on Number of Managers Needing ADP Training in Federal 
Government 



1. Categories of Managers Needing ADP Training in Federal Government 



Heads of Agencies 


120 


Top Staff in Agencies 


1,200 


Program Officers 


4,000 


Managers with Program 
Officers 


20,000 


Field equivalents 
(GS 14-18) 


75,000 



100,320 



Comment ; The above figures equal 3.3^ 
force and apply to positions from the 
down through lower middle management. 



of the total Federal work 
director of an agency 
The chart also includes 



military officer personnel in managerial positions. Training 
one-third of the above personnel is urgent, with the remainder 



very desirable. 



K Comparison of number of individuals 

1. Federal employment GS 15 and above (and equivalent 
pay scales) 



32,691 



2. Field GS 14’s having substantial responsibilities 25,000 



3. Military performing executive duties 



35.000 

=92.691 



Comment : A number of GS 14* s in Washington probably 
also should receive managerial ADP training,- because 
they are ready for promotion. Possibly 5,000-8,000 
persons would be In this category. 

I. Training Meeds - Summary 

From information obtained and discussions held at the conference 
the conclusion clearly emerges that a training/development gap of 
serious proportions exists in the Fec^eral Government. Existing 
training organizations (i.e., CSC, DODCI, ADP Management Training 
Center, etc.) do an excellent job but at current training rates are 
not meeting attrition losses."^ 



4 Presidential Task Force on Career Advancement, Training for Automation 
and Information Processing In the Federal Service (Project E: Trailnlng 
for Specialization) National Bureau of Standards, October, 1966. 
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m. ADP TRAINING IN INDUSTRY 
AND IN HIGHER EDUCATION 

A, Education In Industry 

Industry, like government. Is faced with the same general problems 
of training/development for the systems analyst and other ADP personnel 
as well as for the manager user« The problem in industry, however, is 
somewhat simpler due to three basic reasons: 



1* Any particular or specific industry or company within 
an industry is vastly more specialized in its products 
or services and hence the design and other ADP problems 
to which training is directed can be more narrow in 
scope . 

2. The specialization within companies, reflected in the 
financial considerations, permits a specialized approach 
to training and indoctrination. Moreover, this train- 
ing lends itself more to the on-the-job variety. 

3. The personnel entering the systems analyst field come 
generally better prepared because the two primary 
sources of input are experienced systems analysts and 
col lege graduates. 



In spite of the less complex environment, industry in general and 
the computer industry in particular, is witnessing a new trend - the 
"education explosion." Symptoms of this trend are the 58 percent more 
"in-house" training programs in ADP between 1965-67.^ Equally as 
significant is the vastly accelerating attention and participation by 
top management, including company presidents, in the increasing emphasis 
on training for ADP and other critical company needs. 



^ Farr, Robert N., "EDP Education and the Objectives of Management", 
S ystems, April, 1967, p. 13. An issue devoted to ADP training. 
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Because of the wide variations in the nature and needs of indi- 
vidual firms, no ’‘average" or "typical" training program emerges for 
ADP personnel. This is true to a lesser extent for managerial personnel. 
The needs of this latter group are more clearly defined. Despite con- 
siderable variation in the source of managerial training (e.g., in- 
house, manufacturer, university, consultant) and the time devoted to 
it (typically 2-5 days), some common elements of subject matter generally 
emerge. Briefly, these might include: 

Information Systems - concepts, components, functions, etc. 

How a Computer Operates - operation of components, I/O devices 
Programming and Software - including "hands on" 

Planning - costs, feasibility studies, scheduling 
Implementation - systems design, conversion, new applications 
Personnel and Organization Impact 

Role of Management - support of management, management uses. 

Although no quantitative measures were available for comparison, 
the general impression received from the conference was that the 
depth and scope of training in industry (particularly at the manager- 
ial level) was somewhat less than in the Federal Government, although 
the growing amount of training that does exist in industry reaches 
more people in the organization. 

B. Education in Colleges and Universities 

Keeping pace with industry efforts is the university and college 
corwnunity. Undoubtedly broader in scope than either Industry or 
government, this sector has rapidly accelerated into the fields of 
ADP, systems, computer sciences, management science, and related computer 
areas. In 1962 only four institutions offered degree courses in com- 
puter sciences; today one can earn degrees at over 200 institutions in 
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the United States.^ In the short space of two years (1962-64)^ digital 
computers in use at colleges and universities rose from 397 to 1065. 

This, of course, is not to infer that these computers were being used 
for ADP train ing/educat ion or for instructional purposes. However, 
the reader can predict for himself the enormous increase in computer 
education in the future if the recommendations of the President's Science 

4 

Advisory Committee are even partially adopted. 

More pertinent to the purposes of this report Is the extent to 
which a "systems” type courses are offered at colleges and universities. 
These have been rapidly growing in numbers. A 1966 study conducted by 
the Systems and Procedures Association^ estimates that about 35^ of 
the schools in the country will probably offer a degree in managmeent 
systems by 1971 and about 50^ will offer a minor or major in this area. 

An indication of the growth is the fact that over 50^ of systems courses 
being taught have only been offered within the past three years. The 
more than seventy courses being taught in 1966 were categorized by 



^ Ibid . Of interest is the approach to systems education taken by the 
Carnegie Institute of Technology. As long ago as 1963 doctoral students 
were permitted to major in "systems" in five disciplines, all of which 
had "a common core devoted to techniques of analyzing and synthesizing 
complexity." Reported by Herbert Simon in Koontz, Harold, Toward A 
Unified Theory of Management . McGraw-Hill, 1964, p. 83. 

^ The latest date for which census data were available. Taken from 
Automated Education Handbook, Detroit: Automated Education Center, 

1965, p. VII A 1. 

^ Computers in Higher Education, Report of the President's Science 

Advisory Committee. Washington, D. C. : The White House, February, 1967. 

^ Systems Education in the United States . Cleveland: Systems and 
Procedures Association, 1966, 
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Systems and Procedures Association into: 

1. Introduction to Data Processing 

2. Computer Programming 

3. Management and Genera! Business Systems 

4. Mathematics and Engineering 

The inference, indeed the conclusion, from the foregoing Is that 
in spite of the growth rate of ADP and related education in colleges 
and universities, the number of those trained in these courses is 
accelerating at a rate substantially below the need for the computer 
industry, the Federal Government, and other users. For purposes of 
this report, one can point once again to the need for in-house education 
for federal employees, systems analysts and managers. 
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IV. TRAINING AND DEVELOPMENT OBJECTIVES 

It is axiomatic among educators and people who design or admin- 
ister training programs that the teaching process must be related to 

an objective; some predetermined body of knowledge v/hich it is desired 

1 , 

to transmit or a behavioral change on the part of the learner, it is 
of little use to teach, for example, systems theory to an analyst 
unless it results in application or improvement of some sort. Similarly, 
“hands on*‘ experience for the manager serves no purpose in an of itself 
unless better communications with ADP personnel, improved application 
in his own area, or other benefits accrue. 

Pre-conference instructions and questionnaires asked partici- 
pants to consider the matter of behavioral and other objectives of 
train ing/ development programs. The matter of objectives and the expected 
end result of training were basic to all other discussions and were 
constantly reflected in questions such as these; 

"What do you look for in a systems analyst?" 

"What are the responsibilities of a systems analyst?" 

"How much does a manager need to know and why?" 

"Can you teach an analyst to sell?" 

"How do we close the manager-analyst gap?" 



^ "The objective of training or education can be framed in terms of the 
problems to be solved." Quote from a conference paper delivered by 
Dr. Lawrence Stolurow of Harvard University on Computer Assisted In- 
struction. Similarly stated by Richard Shetler, President of General 
Learning Corporation, "The end process is learning, not teaching* 1^^ 
cannot emphasize this too strongly," See "Innovation in Educat >n. 
Educational Technology . Washington, D.C.: The Aerospace Educat i >.ia I 
Foundation, Spring 1967, p. 9. 
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At that point in the conference where curricula were developed and 
reconsidered, the "image" of a systems analyst and the "educated 
manager" became clear and the objectives of training and development 
were established for purposes of defining subject matter content. These 
objectives are briefly described below. 

A. Development Objective - The Manager 

It is important to understand that when discussing the management 
"student," various complexities and some confusion arise due to many 
levels of managers, their needs for knowledge of automatic data processing, 
and the subject matter to which they might be exposed. Add to this 
the peculiar or specific nature of ADP applications according to their 
subject matter area or agency and the result is proliferation at best. 

This problem was resolved by: distinguishing two managerial groups, 

(a) top management, and (b) other managers. A basic curriculum which 
is common to both groups was then devised. 

For top management, the consensus regarding objectives emerged 
as: 

1. Orient and familiarize the manager to the end that his 
information processing system will be more productive by 

a. Establishing the role of ADP in the organization 

b. Providing tighter control over production of paperwork 

c. Getting the manager involved In problem solving 

(EDP doesn’t solve problems; people do) 

d. Utilizing talents already in the organization 

2a Closing the understanding and appreciation "gap" between 
management and ADP operations.*^ 



^"Almost no single action can be taken that would provide equal return in 
agency operations improvement, than for top officials to adopt a direct, 
favorable position toward ADP use and ADP training. Other ADP actions 
will be ineffective or even unsuccessful if such top management support 
is not forthcoming." Qp.Cit . . p.3. Presidential Task Force on Career 
Advancement. 
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3. Familiarizing the manager with the capability of the 
computer for improving management, effectiveness, and 
productivity through information. 

Regarding lower levels of management the objective became more 
"practical". Whereas the top management objective was more directed 
to an appreciation and understanding of computer technology in relation 
to him and his organization, participants at the conference were of the 
opinion that the lower level manager needed this appreciation also but 
in additiion the training should be directed to how he^ could use computer 
technology in his operations. This was the essence of the recommendation 
although frequent expressions such as "enough training to understand 
the computer technician" or "so he can communicate with the analyst" 
were expressed. 

Subsequently, during curriculum development and in establishing 
specifications for a pilot training project, the foregoing objectives 
were merged into one: 

To provide the manager with the knowledge and skills 
necessary to identify, understand and evaluate the 
potential and performance of the computer system in 
the accomplishment of his operations and mission 
objectives. 

B. Training Objectives - The Systems Analyst 

This report has already stated the CSC description of the duties 
of a systems analyst and pointed out (see page 11) that this was not 
always an adequate description. As stated by conference participants, 
this Inadequacy usually related to his ability to think like a manager 
in designing systems and to his ability to communicate. Summarizing 
this vfew by top ADP executives and consultants were phrases such as: 
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"The Systems Analyst..,^ 

should be trained to use ADP as a management tool." 
shoulid bridge the gap between problem desk (Input) and 
machine configuration (output)", 
must know how to solve problems on the computer." 
must know salesmanship and communication." 
must be a diplomat - know when not to use a computer." 
must understand input/output use of information - not 
throughout." 

An analysis of the discussions and questionnaires from the con- 
ference, in which all in attendance participated, yielded additional 
information that produced an image or profile of the systems analyst. 
Based on this profile then, a training program for Systems Analysts 
should provide for: 



1. The tools and techniques of systems analysis, design, 
and implementation (e.g., hardware, programming, 
documentation, scheduling, work processing, etc.). 

2. Management principles. (Objectives, performance stand- 
ards and the place of information in planning and 
controlling) Needed in order to design information 
systems for management use. 

3. Logic and problem solving. 

4. Systems theory and concepts (integrated systems, 
feedback principles, decision theory, application 
of management science to problem solving and design. 

5. Communications - written and oral, including salesmanship. 
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V. CURRICULA RECa«NDATIONS 

A, Management Deveiopment Program 

Participants and consultants were in basic agreement on the con- 
tent and structure of a curricula for management development programs. 
This curricula with major topical and sub-topical subject matter headings 
is I isted below. 

COURSE FOR FEDERAL GOVERNMENT MANAGERS 
I. HARDWARE AND HOW IT OPERATES 
The workings of a computer 
Capabilities, limitations, and potential 
Input-Output devices 
Software considerations 
File organization and maintenance 
Input-output documentation and display 
New technology 

Objective: To take the mystery out of computers and bring 
an understanding of how they operate and their 
capabilities and limitations. 

I!.. "HANDS ON" EXPERIENCE 

Programming one language 

Writing a program to solve one or more problems 

Other language and sub-routine options 

Objective: To bring an appreciation of how problems can 

be solved with a computer by getting the 
manager involved. 

III. COMPUTER APPLICATIONS 



Basic existing applications 
Advanced applications 
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Future applications 

Objective: To make the manager aware of potential appli- 

cations in his job to the end that he can 
better utilize and participate in ADP appli- 
cations. 

IV. ADP IN THE FEDERAL GOVERNMENT 

Information Economics - costs of ADP, cost-utility tradeoff, 
etc. 

Planning for the System - costs, feasibility, maintenance, 
planning. 

Environment of ADP in the Federal Government - legislation, 
regulations, etc. 

Agency problems and plans 

Objective: To place ADP operations in perspective with 

regard to specific constraints of the Federal 
Government and to describe cost, feasibility, 
and time considerations to the end that the 
manager will participate in and direct systems 
planning and utilization. 

■V. INFORMATION SYSTEMS THEORY 

Management, information, and the computer 

Information systems design 

Analysis and basic design techniques 

Objective: To familiarize the manager with essential nature 

of information and the basic principles and 
theory of analysis and design in order that his 
effectiveness will be Improved by design partici- 
pation. 

VI. DATA PROCESSING AS AN AID TO DECISION MAKING 

Logic, problem definition, and problem solving 
Problem solving with the computer 

Techniques of problem solving - linear programming, operations 
research, simulation, etc. 



Examples and practice in problem solving - cases 



Objective: To show the manager that peopiss, not computers. 

Solve problems. To improve his decision making 
and problem solving ability so that he may become 
more effective with the- computer. 

VII. ORGAN I ZAT I ONAL I MPL1 CAT I ONS OF THE COMPUTER 
impact on personnel and staff 
Organizational changes and manpower consequences 
Role of ADP personnel 
AOP training 

Objective: To get the manager to adopt a favorable position 

toward ADP, ADP training, and to provide an 
environment in which the effectiveness of ADP 
Is optimized. 

B. Systems Analyst Training Program 

During the initial efforts to develop a curriculum for Systems 
Analysts, attempts were made to establish separate curricula for each 
source of Input to the program (programmer, operator, methods analyst, 
college hire, etc.). This approach was found to be not only somewhat 
impractical but on the whole undesirable. A more practical and de- 
sirable approach was determined to be one in which an ideal or "total" 
body of knowledge required by an analyst could be defined, and that 
the prospective trainee, no matter what the source of input, could 
select, by means of a diagnostic process, those modules or elements 
of the "total" curriculum which he required. This, of course, would 
apply as well to the existing analyst who required updating. 

As in the case of the curriculum for management development, a 
remarkable degree of agreement existed among participants and consultants 
regarding the structure and content of the total training, program 
curriculum for the systems analyst. This program is listed below. 




CURRICULUM FOR SYSTEMS ANALYSTS 



I. ANALYSIS AND DESIGN CONCEPTS 

A. PROBLEM SOLVING 

The theory and practice of logic, decision making, and 
creative thinking. The use of these and other problem 
solving principles in the analysis and design of systems. 

B. ORGANIZATION PRINCIPLES 

Classical and contemporary principles of organization 
design and analysis. An understanding of the structure, 
decision centers, information flow, and other organiza- 
tional considerations in systems design. 

C. MANAGEMENT 

The basic functions of management with special emphasis 
on planning and controlling through information systems. 
Consideration and understanding of facilitating the manage- 
ment process with systems. 

D. SYSTEMS PLANNING 

Determining systems objectives and planning time, cost, 
and resource allocations. Design proposals. PERT/CPM 
Input/output considerations. 

E. SYSTEMS THEORY 

Theory of information systems operation and design. 

Control theory. Integrated and total systems concepts. 
Planning and control through information feedback systems. 

F. SYSTEMS EVALUATION 

Measuring efficiency against goalSi- Input-output review 
and review of objectives. 

G. HUMAN INTERACTION IN SYSTEMS 

Gaining acceptance and ’’selling" ADP. The impact of 
automation on personnel. Getting cooperation. Inter- 
personal relationships. Applied psychology. 

H. QUANTITATIVE TECHNIQUES IN SYSTEMS DESIGN 

Application of operations reserach and other management 
science techniques. Formulation of decision rules. 
Simulation and modeling. 
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II. ANALYSIS AND DESIGN TECHNIQUES 

A. SYSTEMS PLANNING 

Network analysis technique for logical structuring of 
planning. Preliminary systems survey. The feasibility 
study. Cost evaluation and analysis. Analysis of time 
requirements. Planning quality elements of the system. 

B. SYSTEMS ANALYSIS AND DESIGN 

Analytical techniques and documentation (work measure- 
ment, flow charting, forms design, source data automation, 
6tc. ) Input/output alternatives. Communications, inter- 
viewing, and selling. Principles of systems design.^ 

C. IMPLEMENTATION AND FOLLOW UP 

Planning, site preparation, personnel, organization, other 
considerations. Training the user. Evaluation and audit. 

III. COMPUTER CONCEPTS AND CAPABILITIES 

A. HARDWARE CHARACTERISTICS 

Mainframe capabi I ity, peripheral equipment remotes and 
linkage, input-output devices, time sharing, on line 
sy terns, etc. 

B. SOFTWARE 

Language options (COBOL, ALGOL, FORTRAN, etc.). Other 
software options (compilers, subroutines, etc.) 

IV. ADDITIONAL SKILL REQUIREMENTS 

A. PROGRAMMING 

Ability to program in one language 

B. QUANTITATIVE TECHNIQUES 

Management science techniques in systems design 

C. COMMUNICATIONS 

Graphics and visual presentations. The oral and written 
staff report. 



new oody of systems principles analogous to principles of management. 
See Ross, Joel E. and Sullivan, J.W., Development of Systems Theory. 
Colorado Springs: Foundation for Administrative Research, 1967. 








C. Additional Considerations - Structure and Time 



Federal . Government officials involved ?n ADP training made it quite 
clear from the beginning of the conference that the subject matter of 
the programs should be designed in modules; standard units of subject 
matter content, clearly defined and distinguishable so that the trainee 
(manager or analyst) would only be required to be instructed in those 
modules which were unfamiliar to him. This approach was variously de- 
scribed as a "menu” or "shopping list" from which the student could 
select those areas in which he required updating or initial exposure. 

The desire to structure the program in this fashion is under- 
standable In view of existing ADP training programs. Almost invariably, 
in the conventional seminar or training program, the audience already 
has varying degrees of familiarity with the subject matter. The result 
is lost time, duplication, boredom, and other unfavorable results. It 
becomes rather clear that the modular approach, particularly if indi- 
vidualized, is more efficient, involves less time, and results in better 
quality instruction. 

Some interest and desire was also expressed in a sequential or 
"building block" concept of instruction. This approach is illustrated 
by the usual method of Instruction in mathematics; arithmetic-algebra- 
calculus-differential equations, etc. This is also the general approach 
to programmed instruction; moving from the simple to the complex. 

The quest ion "whet her the recommended ADP programs should be 
approached in this fashion is a complex one and was largely unresolved 
at the conference. Given the complex environment of the Federal Govern- 
ment (many inputs, various' levels of knowledge), it would be difficult 
at best to design the training program in this manner. 



-34- 



The trainee*s time that should - or could - be devoted to +he 
program was also discussed at length. This amount of time is a function 
of several unknowns such as manager's time, level of knowledge brought 
to the course, and the desire of the agency head* To the extent possible, 
these unknowns were estimated and the level of full time (classroom 
equivalent) effort required by the student to complete the curricula of 
instruction developed was determined to be: 

1. For the manager - between 40 and 80 hours 

2. For the systems analyst - between 120 and 180 hours. 










VI. EDUCATIONAL MEDIA AND ADP TRAINING 



A, Automation and Technology !n Education ^ 

‘ : There can be little doubt that educational technology in the 
United States stands on the threshold of becoming a major industry.^ 
Properly applied, it holds great promise for upgrading and increasing 
the productivity of our educational effort in all areas. Some of the 



promises, problems, and cautions surrounding the application of educa 



tional technology were expressed at 

Aerospace Education Foundation:^ 

Dr. R. Louis Bright 
Associate Commissioner 
U. S. Office of Education 



Dr. Richard Bolt 
Chairman, Bolt, Beranek, 
and Newman 



a recent seminar held by the 



The education market ranks second 
to defense. The problem of educa- 
tion and use of technology requires 
a systems approach between educa- 
tors and industry. 

Technological transformation of the 
classroom is inevitable in the face 
of the population explosion and the 
increasing demand for education. 



Author* s note. It has not been the purpose of the conference or of this 
report to undertake a review of the state of the art in educational tech- 
nology. For those readers who wish to do this, several bibliographic 
and other sources are available. For example, see the annotated biblio- 
graphy (Appendix E)of this report. Also, Hickey, Albert E., Computer- 
Assisted Instruction, A Survey of the Literature . Newburyport: ENTELEK, 
Inc., 1967 (ONR Contr. 4757(00) • Also, Automated Education Handbook. 
Detroit: Automated Education Center, 1965. Also, "Innovation in Educa- 
tion," Educational Technology, Spring 1967. Annotated references to 
particular subjects may also be obtained by the Defense Documentation 
Center. 

2 

"The American economy was built around the railroads in the last half of 
the 19th century, around the automobile in the first two-thirds of this 
century, and it will be built around education in the balance of this 
century." Witness before hearings of Subcommittee on Economic Progress of 
the Joint Economic Committee. U.S. Cong., Automation and Technology In 
Education, 89th Opngress, 2nd Session, August, 1966. 



^Educational Technology, Op. C!t. 
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Dr. Calvin Gross 

Dean, School of Education 

University of Missouri 



Richard Shetler 

President, General Learning 

Corporation 



Urged acceptance of the cost/effec- 
tiveness concept in education and 
how increased capital investment in 
technology can achieve greater pro- 
ductivity and effectiveness. 

We should be seeking ’'validated 
learning systems” achieved through a 
system including equipment, pro- 
cedures, materials and personnel. 



■ In spite of its great promises and prospects, educational tech- 
nology has had something less than widespread, adoption in the educational 
environment. Computer assisted instruction, in particular, has been 



confined largely to the laboratory or experimental situation. Educational 
’’software” is generally not available and much equipment is highly de- 



velopmental and experimental. The Subcommittee on Economic Progress of 



the Joint Economic Committee has outlined the generally accepted factors 
which will govern the potential contribution of educational technology 



in the future: 



1. Effectiveness of research in learning theory and its 
application to the development of education; 

2. Improvement of curriculum programming, particularly 

in respect to defining and meeting educational objectives; 

3. Organization of our school systems and intelligent 
planning of curriculum; 

4. More effective use of teachers; and 

5. Recognition on the part of teachers and educators ^ 

of the great potential in the new educational technology. 

It appears that any ’’across the board” adoption and use of edu- 
cational technology does, indeed, face many problems. However, with 
regard to the objectives of the conference on which this paper reports. 



U. S. Congress, Automation and Technology in Education, Op . C i t . . p. 8 



o 

ERIC 



-37- 



the conclusion emerges that most of the prerequisities to application 
of technology for ADP training are solved and present. Given the objec- 
tives, environment, curriculum, and other prerequisites to ADP training 
in the Federal Government, there v/as no expression of substantial reason 
why such training should not proceed. 

B. Present and Future Use of Media in ADP Training -Conferees 

In this section of the report the reactions and plans of conferees 
regarding use of educational technology will be reported. Because of 
the different approaches and interests In the problem, conferees will 
be reported in three groups: (1) equipment manufacturers, (2) ADP 
executives and consultants, and (3) other conferees. 

Representatives from educational equipment manufacturers (Appendix II) 
were unanimous in their expectations and in their praise of multi-media 
and educational technology applications in the near and longer term. 

Each company planned for extensive marketing efforts and the forecast ; 
of sales and profits were substantial. As a result of these plans and 
forecasts each company had undertaken reorganization and other efforts 
to insure that top level corporate attention was given to this growing 
market. 

Each agreed that the industry and its applications were complex. 

The answer lies not in the development and marketing of specific items 
of hardware but in the systems approach; the "orchestration" of media, 
software, and educator’s use and acceptance. A variety of talents and 
resources are necessary in the systems approach to insure the coordina- 
tion and utilization of an educational ’package' instead of the piece- 
meal approach taken to date. And yet, events are moving so rapidly that 
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the user cannot wait for standardization and proven validity in either 
hardware or software. Indeed, to wait for or enforce standardization 
and maturity in development would mean stifling creativity and experi- 
mentation; elements that are so badly needed in the current state of 
development. 

Each agreed that computer assisted instruction (CAD was much 
broader In its application than the simple tutorial dialogue envisaged 
by the general public and most educators. CAI includes simulation, 
problem solving, and the automated use of all forms of media. 

In many respects, CAI Is a broader and more sophisticated appli- 
cation of programmed instruction (PI) and each participant was lavish 
in his praise for PI. Each has extensively utilized PI or PI with other 
forms of visual media for inter-company training, both in ADP and in 
other subject matter areas. It was agreed that experience in PI (written) 
provides the experience and head start necessary to move into CAI. Only 
one manufacturer (IBM) had made significant use of CAI for ADP training. 
Results had not been sufficiently validated for release to the general 
public although one participant. Dr. Sylvia Charp of the Philadelphia 
School System, reported very enthusiastic response from the use of the 
program on an experimental basis In Philadelphia. 

Without exception, representatives of equipment manufacturers 
reported the number one problem as software; not machine' languages but 
the shortage of subject matter specialists who could define educational 
objectives, arrange instructional material in programmed format, or 
otherwise program curriculum material for application and use on 
technological hardware - whether CAI or otherwise. 
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without exception, manufacturer's representatives were enthusiastic 
about the use of muiti -media technology to ADP training over the near 
and longer term. Most predicted that software costs, both in time and 
effort, could and would be reduced with new knowledge and technology. 

In general, the opinions were that CAI in its broadest sense and in its 
several modes contained the answer to individualized, effective ADP 
training. 

C. ADP Executives and Consultants 

With two or three exceptions, ADP executives and consultants (See 
Appendix II) reported that their ADP training techniques were conventional 
(sometimes described as primitive) and that media consisted largely of 
chalk and blackboard. All agreed that something must be done to fill 
the training gap but individual companies had done little, due to the 
non-availability of curriculum material programmed for media and 
the expense of developing it on an individual basis. 

Most of the participants reported that written PI had been used 
and uniformly good results had been obtained when properly supervised. 

Some had utilized TV tapes or motion pictures where these have been 
aval I able. 

Each member of this group singled out managerial training as a 
special case. Becuase of the scarcity of time available to managers and 
their reluctance to undergo classroom or formalized instruction, some 
type of media approach is devised to get and hold their attention. This 
approach was variously deseribed as ’’schmaltz" or "show businss" in 
order to get the manager involved. Slides, movies, television, modeling, 
graphics and other tools were described as in use and one executive 
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reported that the well designed program using these media can cut the 
time required down by a factor of ten to one. There Is, of course, no 

reason why this approach would not work for other groups (i.e., systems 
analysts) as well. 

An outstanding exception to the mediocrity of structured in-house 
company training is the case of the American Telephone and Telegraph 
Company in New York City. Terminal use for enterprise modeling, simu- 
lation, programming and other subject matter areas is utilized in this 

company as part of a formalized program utilizing the latest in educa- 
tional media. 

D. Other Conference Participants 

Workshop participants represented a wide range and variety of 
subject matter specialists, curriculum and education specialists, PI 
and CAI technical specialists, and Federal agency officials concerned 
with ADP training, (See Appendix II), It was not surprising to find an 
equally wide range of experiences with various educational media. About 
half of the participants (many from Federal Government) stated that ADP 
training in their organization consisted largely of conventional lectures 
and discussions sometimes accompanied by slides or other visual aids. 
Other incidents of media described as successful were written Pi, pro- 
grammed films, television, and simulation, Exceptrfor Federal Government 
participants, virtually all the remainder had plans for increased use of 

media in the future, Many were planning for increased utilization of 
PI and CAI , 

No consensus developed among participants regarding the use of 
educational technology for AOP training in the future. This is to be 
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expected In view of the complexity of the environment in which ADP 
training takes place in the Federal Government. As one participant 
stated, "You can’t determine reliability and validity of specific media 
for particular purposes until objectives of the program are clearly 
identified. Time, money, and place constraints, as well as variability 
of individuals entering the program are ai! more significant than the 
med i a to be ut i I i zed . " 

Almost aii participants felt that the use of media (film, film 
strip, slides, audio tapes, television, simulation, computers, PI, 

CAi, etc.) improved instruction. The majority felt that CAI in its 
broadest sense held great promise for ADP training, provided adequate 
software became available. There was general agreement that if one could 
ignore cost considerations (one not adequately discussed at the con- 
ference), some form of CAI utilizing multi-media was probably the best 
approach to ADP training in the Federal Government if the objectives of 
indvidual ized mass instruction with modular design of subject matter 
were to be achieved. The general acceptability of multi-media and CAI 
for ADP training is reflected in the recommended specif ici at Ions for a 
pilot project contained in Appendix I. 

E. The Problem of "Bra inware" 

No other single Item of discussion surrounding educational tech- 
nology was more prevalent than the problem of "software" - that effort 
required to convert subject matter to a form which can be retrieved by 
the learner in a manner that will achieve the educational objective. 

This problem remained central to all considerations of utilizing CAI 
and other media. Many felt that the term, "softv/are", having the 
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connotatfon of programming the hardware, did not adequately express 
the task or resource required for programming the course content. The 
term "courseware” or "brainware" therefore Is more appropriate for this 
difficult task which so often is the bottleneck In adequate utilization 
of educational hardware. 

There is little question that the foregoing problem is central to 
any consideration surrounding the development of an ADP training program 
in the Federal Government. It was well expressed by the representatives 
of one of the nation’s largest companies involved in educational tech- 
nology, "The preparation of curriculum or instructional programs is a 
long process requiring careful professional effort. It can only be 
handled by a group of professionals including teachers, media specialists, 
technologists, etc., operating in a carefully planned and scheduled de- 
developed program. ’Hardware’ is basically not a problem, but ’software’ 
is the intelligence of the system and therefore the critical item. One 
should be particularly wary of ’6IG0’ in instructional systems." 
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A. Specification for Requests for Proposal (RFP) 

Reported herein is the consensus of expert opinion regarding the 
general description of and specifications to be included in an RFP for a 
pilot project to conduct ADP training in the Federal Government. 



B. General Nature of the Project 

Establish a training facility in the V/ashington, D.C. area designed 
to serve as a pilot effort in the use of individualized instructional 
media and hardware. This facility will serve as a prototype of similar 
facilities to be located in other key cities in the United States and 
in other Washington locations. 

The project involves the development of two training packages, 
utilizing the curriculums listed in the main body of this report. The 
two groups of participants of 25-150 each are (a) Federal Government 



managers and (b) systems analysts, 
training are: 



Management Course 

To provide the manager with 
the knowledge and skills 
necessary to Identify, under- 
stand and evaluate the poten- 
tial and performance of the 
computer in the accomplishment 
of his operations and organ- 
izational mission objectives. 



The specific objectives of the 



Systems Analyst Course 

To provide those individuals who 
have adequate background in com- 
puter programming with the know- 
ledge and skills necessary to 
contribute meaningfully to all 
basic phases of a systems project 
under competent technical super- 
vision. 



The courses should be aimed, as a general policy, towards indi- 
vidualized instruction utilizing the most advanced but feasible and valid 
instructional media, including computer assisted instruction. To this 
end the contractor shall explore and report on ail media of instruction 
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and identify, specify, and locate the equipment necessary to conduct 
two pilot demonstration projects. 

Participants in the pilot courses will be selected from a variety 



of agencies and activities by the d' 
ticipants v;ill be made according to 

Management Course 

1) GS-14 or above, or equiva- 
lent 

2) Persons who are not now and 
have not been ‘’computer 
professionals” 

3) No prior computer knowledge 
is assumed. 

4) Participants will be mature 
individuals; however, no 
formal educational back- 
ground beyond high school 
will be requi red. 



Participants wi 1 1 have vary ing 



Ignated agency. Selection of par 
he following criteria; 



Systems Analysis Course 

1) Experience or hands-on train- 
ing in computer programming 

2) Passing of a mathematical apti- 
tude test equivalent to first 
year college algebra 

3) No single pattern of Federal 
experience will be assumed; 
younger students will tend to 
have recent academic back- 
grounds - however, students 
will be selected on the basis 
of general experience. 



rees of education, experience. 



and academic background. Participants In the systems analyst course will 
be from a variety of input sources, including programmers, methods 



analysts, systems analysts requiring update, and new college hires. 

It can be assumed that the level of full time (classroom equivalent) 
effort required by the student to complete these courses should be: 



a) Managers course - between 40 and 80 hours 

b) Systems analyst course - betv/een 120-180 hours. 
(Students in this course can be required to per- 
form additional homework assignments) 

C. Specifications for Instructional Program 

1. Instructional programs will be prepared to facilitate the 
curriculums outlined in the main body of this report (pp. 28-33) 
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2. Instructional programs v/ill be organized into modular units to 
permit Independent selection of only those units in which the student 
is not proficient by reason of education or training. 

3. To implement the instructional program and the concept of indi- 
vidualization a system will be devised which will Indicate the choice and 
sequence of modules appropriate for students with various levels of 
knowledge at entry and various performance requirements of the student. 
This sytem may include a tabular description or matrix of selection ■ 
criteria (e.g., IQ, mathematical aptitude, education, experience, etc.) 
for each module and a procedure to test criteria. The system is 
diagnostic in the sense that the program director can determine the 
participant's entry level in the modular program. 

4. Instructional program development will include recommendations 
of methodology for each module, including media alternatives (PI, texts, 
film, CAI, etc.) and pedagogical techniques (case history, simulation, 
etc. ). 

5. Where the contractor bids on instructional programs only, such 
bids shall include the actual instructional materials ready to be written 
in problem-oriented language for machine processing (in the case of 

CAI) or prepared on other similar appropriate media. 

D. Specifications for Media/Hardware 

1. All hardware should be adaptable to both central and remote 
utilization and capable of being expanded in the number of students 
that can be accommodated. 

2. To the extent that new equipment is developed for the pilot 
project, it should be approved before utilization in such areas as human 
engineering for utilization to promote a good learning environment. 
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3. Primary consideration shall be given to equipment presently 
available within the Federal Government. 

4. Contractors proposals will recommend specific media or com- 
binations thereof for instructional module. One or more areas should be 
devoted to Individual Tzed CAI. 

5. Classroom instruction with remote units in order to provide 

for individual consoles in group instruction sessions should be developed 
in one or more areas. For example, a lecture in simulation could be 
supplemented with individual student exploration of the effects of 
various inputs to a model. 

6. Provision must be made for the capability to employ conventional 
training aids such as chalkboard, screens, closed circuit TV. 

7. The following general characterlsitcs are considered desirable 
in the instructional system. However, cost-effectiveness tradeoffs 
should be examined and reported. 

a. Multi -media presentations 

b. Computer controlled 

c. Human engineering, design criteria 

d. Mechanical equipment ''sign on" and student identification 
(key for machine and identification badge for student) 

e. Modular design. Major components should be sufficiently 
independent to facilitate replacement upon failure to 
minimize ’down time’. 

f. Self checking. Equipment design to provide sensing devices 
to detect component failure and such information to be 
relayed to a central source for monitoring equipment per- 
formance. This design to be augmented by a student trouble 
indicator to serve the same purpose. 



-48- 



g. Individual media requirements 
Manual 

^Full typewriter keyboard 

Light pen (or provision for acting upon student 
touching of screen) 

5 to 10- key response keyboard 

Audio 

Speaker and head set with stereo capability 
Audio level under program and manual student control 
Norm/compressed speech mode switch 
Speech compression level under program and student 
manual control 

Voice 

Provision for voice response input for program 
control 

Voice recording and playback under program control 
Visual 

CRT - provision for rear projection of film media 
on to face of CRT. Brightness under program and 
student manual control. Closed circuit TV coupling, 
random access video tape image selection 

Rear Projection Screen 

Projection device to be cartridge loaded by student 
(film or slide) with capability of randomly project- 
ing individual frames, switching picture to either 
screen or through CRT to the tube face. Projection 
speed to be under program control. 

*'Fixed” Instruction Panel 

Capable of displaying minimum of ten lines of 25 
characters each. Illumination of appropriate line 
under program control and lines so constructed that 
they may be modified in content to fit authors^ 
requi rements. 

h. Acceptance and selection criteria 

([Contractor should be able to demonstrate the reliable 
functioning of all system components under conditions of 
simulated usage. Testing should be for a period of no 
less than 100 hours, the last 40 hours of which must have 
at least 95^ up-time with no more than 10 failures overall 
and of these 6 must be for periods of less than 5 minutes. 

Qualitative factors to be considered in the evaluation of 
equipment performance. Given simi lar equipment design and 
functioning, preference in selection will favor those con- 
tractors whose equipment demonstrates superior quality in 
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h. (continued) 

the following areas of human engineering (to include 
esthetics) 

Low noise levels 

Sharpness/resolution of image projection 
Audio outputs 

Function layout of display elements and controls 
Rapidity of component response 

System restart in the advent of computer and/or power 
fai lure 



E. Contractor Criteria 

1. Contractor shall have demonstrated competence in curriculum 
development, educational media, computer equipment, software development. 

2. List complete qualification data of employees, both adminis- 
trative and professional and technical on the project. 

3. Any substitution of personnel In Item 2 (above) will be with 
prior approval of the contracting officer. 



F. Time Schedule 



The time schedule for design and implementation of the pilot 
project is shown below. 

Event Cumulative calendar days from 

Request for Proposal 

RFP 0 



Expression of bidder interest 


30 


Receipt of proposals 


104 


Selection of contractor 


194 


Award signed contract 


224 


Preliminary report of contractor 


254 


Pilot installation availability 


700 


(including declaration by contractor 
of readiness to accept student groups) 
Commence pilot courses 


790 



Contractor’s final report will be due 
within 60 days of completion of pilot 
course 
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Proposals wM ! Include milestone reports covering minimally: 

1. detailed outline of curriculum 

2. media 

3. software 

In addition, monthly project reports will be required of the contractor. 

or two expert cons ultants should be engaged to assist the contractinn 
Officer in the supervision of the contract . 
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APPENDIX II 



Conference Participants 



PANEL A 

Panel of Top Government ADP Executives 

Miss Ann Lamb 
Bureau of the Budget 
Washington, D.C. 

Mr. Edward Dwyer 

Assistant Commissioner, ADP Management Services 
General Services Administration 
Washington, D.C. 

Mr. John P. Eberhart 

Director, Institute for Applied Technology 
National Bureau of Standards 
Washington, D.C. 

Mr. Carl Clewlow 

Deputy Under Secretary of Defense 
Department of Defense 
Washington, D.C. 

Mr. Charles Sparks 

Deputy Director 

Bureau of Management Services 

Civil Service Commission 

Washington, D.C. 



PANEL B 

Panel of Educational Technology Manufacturers 

Dr. Robert D. Gates 

Director, Educational Operations 

Philco-Ford Corporation 

Fort Washington, Pennsylvania 

Mr. William Greiner 
Manager, State and Local Marketing 
UNIVAC, Division of Sperry Rand Corporation 
Washington, D.C. 

Dr. Edward L. Katzenbach , Jr. 

Vice President 
Raytheon Company 
Lexington, Massachusetts 
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Mr. Len Muller 

Director, Instructional Systems Development 
IBM Corporation 
Armonk, New York 

Mr. William Kilroy 
RCA Instructional Systems 
Radio Corporation of America 
Palo Alto, California 



PANEL C 

Panel of Consultant Firms - ADP Training 

Mr. Dave Allison 
Brandon Applied Systems 
30 E. 42nd Street 
New York, New York 

Mr. Frank Reilly 
Booz, Allen & Hamilton 
Management Consultants 
Cafritz Building 
1625 Eye Street, N.W. 

Washington, D.C. 

Mr. Dick Sprague 
Touche, Ross & Bailey 
60 East 42nd Street 
New York, New York 

Mr. Jack Veale 
Technical Operations 
17J1 N. Kent 
Washington, D.C. 



PANEL D 

Panel of Industrial Firms - ADP Executives 

Mr. Lyle Brewer 
Director, Systems 
Eastman Kodak 
Rochester, New York 

Mr. Robert Parsons 
Vice President, Systems 
Eastern Airlines 
Miami, Florida 

Mr. H. J. McMains 

Director, Analytical Support Center 
American Telephone and Telegraph 
150 Williams Street - Room 1116 
New York, New York 
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SPECIAL SPEAKERS 



Dr. .Gabriel 0*Fiesh 

Director of the Center for Educational Technology 
Catholic University 
Washington, D.C. 

TITLE: "Vital Concepts of PI and CAX for ADP Training" 



Dr. Lawrence Stolurow 
Director, Computer Center 
Harvard University 
Boston, Massachusetts 

TITLE: "The Process of Machine-Learning and Employee 

Training" 



Subject Matter Specialists 



Dr. William Atchison 
Director, Computer Sciences Center 
University of Maryland 
College Park, Maryland 

Dr. Sylvia Charp 

Director of Instructional Systems 
Philadelphia City Schools 
21st and Parkway Streets 
Philadelphia, Pennsylvania 19103 

Dr. Ralph Van Dusseldorp 

Associate Superintendent 

State Department of Public Instruction 

University of Iowa 

lox^^a City, Iowa 

Dr. Malcolm Gotterer 
Professor of Business Administration 
Pennsylvania State University 
University Park, Pennsylvania 

Dr. Herman Limberg 

Director oi Management Reporting 

Office of Adiainistration 

Office of the Mayor 

New York, New York 

Mr. Fred Simmons 
Director of Computer Technology 
University of West Florida 
Pensacola, Florida 
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Curriculum and Education Specialists 

Dr. William Katzenmeyer 
Department of Education 
Duke University 
Durham, North Carolina 

Mr. Walter Corvine 
Assistant Professor 
Chicago State College 
6800 South Stewart 
Chicago, Illinois 

Dr. David Shirley, Professor 

College of Business and Public Administration 

University of Arizona 

Tucson, Arizona 



PI and CAI Technical Specialists 

Dr. Donald Hartford 
Associate Director 
Computing Center 
Auburn University 
Auburn, Alabama 

Dr. Lawrence Stolurow 
Director, Computer Center 
Harvard University 
Cambridge, Massachusetts 



Federal Agency Participants 

Mr. Park Anderson 
Assistant Director 
ADP Training Center 
Civil Service Commission 

Mr. Frederick Dyer 

Office of Civil and Manpower Management 
Navy Department 

Mr. Howard Gammon 
Assistant to Director 

Center for Computer Science and Technology 
National Bureau of Standards 

Mr. Richard A. Gay 

Chief, Systems Management Staff, Personnel 
Public Health Service 



Dr. Chester L. Guthrie 
Deputy Assistant Archivist 
for Records Management 
National Archives and Record Service 

Mr. Joseph Lowell 
Director, ADE Training Center 
Office of Career Development 
Civil Service Commission 

Dr. Richard Otte 

Project Officer 

Human Resource Branch 

Division of Adult Vocational Research 

U. S. Office of Education 

Dr. Hugh Scott 

Automation and Manpower Policy Officer 
Civil Service Commission 

Mr. William J. Shickler 
Office of Manpower Policy 
Evaluation and Research 
U. S. Department of Labor 
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APPENDIX III. 

ABSTPJ^XT OF RESOURCE I5APERS 
DELIVERED AT CONFERENCE 




COllPUTER ASSISTED IlISTRUCTION 



Abstract of a paper delivered by 
Dr. Lawrence M. Stolurow of Harvard 
University. 

There is a real place for CAI in the spectrum of training problems xorith 
which we are faced. The problem is to see where it fits and hoi7 it can be put to 
use. It might be that we will try CAI and find that this is not the best way to go 
but this is not unique to CAI. 

We should think of CAI as a dual purpose investment; one, to determine its 
limitations and capabilities, and secondly, to get mileage out of it in instructing 
students. In other words, it should not be Investigated in a laboratory remote from 
the real situation. The ideal places for initial study are those existing systems 
(e.g. , airline scheduling) capable of time sharing, multiple access, and teleprocsfes^ 
easing. 

CAI is not programmed instruction. The former is computerized, machine 
augmented instruction and is a very different concept than PI. Many people make 
the mistake of thinking they can dump existing PI onto a system but there is very 
little advantage from making such a simple transfer. 

Our concept at Harvard is that CAI should provide a means of pretesting 
and developing instructional materials. It may turn out that the marketing or use 
of these materials may take a very different form. PI, of course, does not have 
this capability. 

The basic concept in machine augmented instruction is that it is individ- 
ualized — that is its main purpose in terms of application. Moreover, a computer 
based system has two capabilities not present in other media;, logic and memory which 
provides essentially a nervous system for whatever media you want to employ for in- 
struction. This leads to the point that the two very basic conditions that you want 
to focus on in thinking of machine augmented instruction are first that its multi- 
media — and it also has a multi-mode capability. 



To demonstrate what is meant by muiti-media, we can look at the following 
slides. These consoles are primitive and indicate the limited nature of those 
which are available. Much more human engineering needs to be done before we have a 
learning environment. Existing consoles are adaptations of input/output equipment 
designed for other purposes. 

(The speaker narrated the following slides and commented on each) • 

Slide No. 1 - The console is a small CRT and keyboard with an IBM 
360/50 as central processor, used at the University 
of California to teach mathematics. The instruction 
burden is primarily carried on by the teacher but 
individualization comes in by allowing the student 
to see a display of his ovTn concepts. For example, 
in teaching a mathematical function, the student can 
enter the variations in the parameter and see what 
the function looks like. 

Slide No. 2 - A system individually designed for CAI but concerned 

primarily with display rather than response. Left 

hand display is a rear view projection of a 35 MM 

film, and at the right is a fixed message display 
for messages repeatedly 

used. The film display is a step toward the kind of 
learning environment which is needed because it gives 
you cheap storage. 

Slide No. 3 - An IBM 1050 typewriter, capable of operation in tele- 
processing mode. To the left is an additional RPQ. 

The top segment is a screen for a random access slide 
projector in synchrony with text display. This is 
essentially an AV console. 



Slide No. 4 - A Philco-Ford, a standard TV tube modified for C?T display 



x?hich permits showing of tapes , TV displays or hook up to 
live broadcasting. Also adapted is a keyboard and I ight 
pen response. The light pen capability adds quite a bit 
but it also adds to the cost due to the coding, programming, 
and delay in materials preparation. 

Slide No. 5 - The console for the 1500 system. It contains the key- 
board, the CRT, an image projector, audio, the light pen, 
and a microphone for recording vocal responses. 

All of the foregoing are a pass at the problem rather than solving it. 

None have any real human engineering behind them and this remains one of the sore 
spots in the whole input/output problem for CAT or machine augmented instruction. 
However, the main point was to show that CAI is multi-m.edia. If this multi-media 
integration capability can be put together in an appropriate educational package, 
then we are moving ahead in terms of our training requirements. 

The second major point to make about CAI or machine .augmented instruction 
is that it is multi-mode. The first mode is the problem-solving mode. 

''By problem solving, I am saying that we are dealing with a situation in 
which the capabilities of the computer as a calculating system are being 
employed with each individual sitting at a console putting in his data, 
defining his processing program and getting the system to perform what 
it has been designed to perform, namely calcuiAtion. BASIC is a language 
which is being used on the GE systems. It’s readily learned by students. 
It’s simple enough to take care of most of their physics, math, chemistry 
problems, statistics, etc. Essentially the console is being used as an 
elaborate desk calculator. The faculty or staff in an educational insti- 
tution needs to know very little more about computers, languages, about 
time-sharing, etc., than they now know. You don’t have to build up your 
^tetoUitional staff. You simply let your students learn a language and 
the staff can assign the usual problems and maybe more of them because 
the students now have a capability of handling more and getting more 
problems solved.'* 

’’Drill and practice is another mode. In this mode the instructional 
staff has to make decisions about what kind of drills and what kind of 
practice the students are going to need in support of the instructional 
effort. And then they have to work with some computer types to generate 
these materials or write themselves , in which case there is greater in- 
structor involvement, teacher involvement, or author involvement in the 
process. The student on the other hand obviously responds in natural 
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language to natural language displays and so he needs to learn very 
in order to use such a system. The main advantages are that the student 
can be given specific and tailored practice to his particular needs. 

Once these needs are deteririned by some other means the system can support 
the skill development.’" 

■•‘The third mode is one which is called inquiry mode which is your 
mation-retrieval mode of the application of the system which has differ- 
ent system requirements. You need algorithims: you need some kind of 
analysis of your data base; you have to set up your files; you have to 
maintain files. An example would be the application at SDC where they 
took the Golden Book Encyclopedia, loaded the information in it on t *e 
system and then set up research algor ithims. One can ask why it rains 
and what ever the Golden Book said in answer to that question will be 
displayed on a console to the student. 

"The fourth mode is the tutorial mode. In the tutorial mode we are 
taking the application of machine augmented instruction, where the 
responsibility for an instructional program has been assumed by somebody 
who is called an author or educational programmer, where he is responsi- 
ble for taking individuals from some point of minimum level of confidence 
and moving them to some set of objectives in terms of improved confidence. 
The logic for doing this — the v?ay in which one formulates what 
knoxm as teaching strategies — is one of the problems. Another problem 
is the allocation of materials to media — decisions have to be made 
about this, as V7ell as the strategies.' 

“The fifth mode is probably the one that is the most prilmitive but 
potentially the most important mode to us, particularly at this stage of 
the game. This is what is called the author mode, where we want to 
develop the capability of the system to generate instructional material 
to actually produce instructional materials, from general specifications. 
The question is what kind of modules, what kind of algorithiras do we 
need in order to combine these modules to provide texts that would sup- 
port a particular instructional objective and generate it so that it 
doesn’t have to be prepared beforehand and then loaded on the system, 
but rather the modules could be loaded and economy achieved in this way. 

Specific examples of what we have done so fari^in this mode are to generate 
syllogisms in the support of a logic course where we would not have to 
write every syllogism in advance, in fact, not know what syllogism the 
student is going to get. All we know is that at this point in time, he 
is to get a syllogism of a certain type in either abstract or concrete 
form. So if it’s abstract he gets it in a’s and b's or all x’s, y s ^ 
and so on. If it’s concrete, he will get a verbal description instead. 

The approach to tutorial instruction is in two phases, the pretutorial 
and the tutorial. 

The pretutorial phase (shown in Figureil) must be able to receive the in- 
puts of student characteristics and specific behavioral objectives of instruction so 
that a teaching program can be selected. 
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**The three alternative search strategies are indicated, the most inter- 
esting one of which is at the bottom. The top one is where you come up 
with more than one possible program for the individual and have to use 
economic criteria for reducing it to one. The middle is where you get 
one program. The bottom is where you come up with none but you have a 
training requirement or a quota and you have to raake some changes. One 
specifies that the change in the entry level is one possibility — if 
it’s a statistics course, you might find that he is deficient in his 
"algebra and therefore you give him an algebra refresher before you al- 
low him to take the statistics course. That’s indicated by the top 
flow through the diagram. You might accept a greater risk and accept 
individuals with lower entry levels who have a higher aptitude in the 
particular relevant area and increase your risk, but also increase your 
take in terms of potentially trained people. The other is to change 
the set of topics that you’re going to require, the time allotment, or 
the final performance level. One can make any one of these changes or 
a combination of them and set them up in a priority schedule, and the 
system would automatically be able to process Individual students v/hen 
they come for instruction and take care of them so that you vjould meet 
your quotas, ilow that’s the pretutorial phase. A cai^ability for doing 
these things needs to be in a total system.*' 
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The Tutorial instructional system 
Figure 2. 



Figure 2. 

Xatorlal- Process . Figure 2, concerns the basic elements and relation- 
ships in the tutorial process. Feedback includes the idea of reinforcement and, 
therefore, is an explicit recognition of the need to make selected events contin- 
gent upon response. Feedback is recognized as a stimulus ^vent that follows re^ 
sponse, as distinguished from the cue and eliciting stimulus functions. Feedback 
ife a more descriptive term since school situations involve both informational con- 
tingencies and reinforcement contingencies. 

The performance standard (Flgue 2) is a critical function in the making 
of decisions in experimental studies of learning. Teaching can no longer allow 
performance standards to go unnoticed as a critical part of the definition of the 
learning task; their function needs to be made explicit. 
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Performance standards are the various principles used to generate the 
criteria employed in making decisions about the correctness of each response. 
Standards vary with school grade and often V7ith a student* s ability. Teachers 
impose different standards upon differs$nt students in a class and in this way define 
different tasks for these students. 

Performance standards used by the teacher at an early period can be taught 
to students at later periods. This makes performance standards a part of the materi- 
al covered in a course and thus available to the students who can later use them to 
evaluate their own performance. The student who is taught rules of grammar, for 
example, can apply these to his own writing. Once he does this, he becomes his own 
teacher by supplying himself with knowledge of results and, if he verbalizes the 
rule when he corrects his own work, even with information feedback. This links the 
rule to the cues of the material and establishes an even more effective association 
for later use. 
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"PROGRAMMED INSTRUCTION" 

Abstract of a paper presented by 
Dr. Gabriel 0* Flesh, Director of 
Center for Educational Technology 
Catholic University, Washington, D. C. 



‘‘I am very gratified to tell you that I received a letter several 
days ago containing a document signed by the Commander of the Air 
Training Command, the largest training establishment in our nation, 
saying that the concept of program instruction has proved itself 
so effective in the research and development staffs it V7±ll be 
applied across the board to all the air training programs." 

"The basic problem we must address ourselves to in the new field 
of science of education is — how do we use the psychology we are 
talking about to train the technologists? We must concentrate our 
limited resources on developing them to turn out the massive amounts 
of materials in our society which are designed to produce systems 
which will be adopted to the accelerating requirements of our needs. 
Education faces a crisis . The sheer force of the mushrooming popu- 
lation makes our traditional educational system inadequate. ...The 
measures which we have been forced to take to meet today’s needs 
will be totally inad6fquate tomorrow." 

"The rapid accumulation of information in almost every discipline 
and endeavor has forced a few of us in the field of education to 
search for new methods of acquiring, assembling, analyzing, and 
disseminating the almost overwhelming new knowledge of our age. 

The development of teokniques and devices such as educational 
television, teaching machines, audiovisual communication, and above 
all the programmed instruction process itself has been hailed as 
revolutionary and capable of solving problems .dissociated with the 
knowledge explosion." 

"These devices and aids are considered by some of us not to be 
revolutionary in themselves but only to the needs of education. It 
is great for correcting our teacher shortages, illiteracy, etc. 

It provides us with a means of revising education. It provides 
us with a means of increasing the effectiveness of our curriculum. 

It helps us in the field of learning theory, in the process by which 
human beings learn. It is becoming apparent to us that the develop- 
ment of packaged learning systems - the same assembly line process 
that Bessemer brought to steel and Ford brought to the automo- 
bile - unless we allow this to take place, it is doubtful we will 
be able to solve the rapidly accelerated problems of our society." 
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Increasing technological change, exploding world population, accel 
^I0n| ^o^ui rates, the need for commencing better education earlier 
in life — these are illustrationsof these problems. 

”An educated citizenry is one of the most effective tools we have 
for economic and political growth and development. 

Statesmen, economists, and many other groups of national and in- 
ternational leadership are recognizing that the only thing more 
expensive than education is the lack of it. 

These problems cannot be reliably solved as long as current educa- 
tional conditions exist; they are ntt going to be reliably solved 
as long as we invest in the same kind of economic solution. And 
in a society that puts moreramoney into the programming of our 
major TV networks in one week than we do in all educational TV in 
one year.’" 

"In order to break this frustrating circle, we need a new, vigor- 
ous approach. 

We need something to overcome the traditional constraints of 
making the best instruction possible available to every individual 
al — including the teachers. Through the process of programmed 
instruction and computer-assisted institutional facilities, this 
possibility can become a reality...." 

"Through programmed instruction, the best tutorial methods can 
be packaged and mass-produced for students. Through packaged 
tutorial learning systems the few master teachers and their 
skills and the teachers of teachers we have in our society can 
be relieved of the many dull routine aspects of the didactic 
process and devote their creative energies to those precious in- 
terpersonal moments between student and teacher and between 
teacher and student- teacher." 

"I dropped into one of our western cities not long ago and saw 
the most amazing machine I ever saw in my life — 25 knobs on it. 
This was a coffee vending — not education vending. It had a 
knob for sugar and no cream, cream and no sugar, heavy sugar and 
cream, light sugar and light cream, etc. Yet students all go in 
the classrooms and get the same black coffee curriculums. We 
can't seem to assemble the same resources bo all of our people 
can have continuous education." 

"The education profession must retool immediately to cope effec- 
tively with the everchanging educational needs of our people. 
Innovation is needed in the retooling process and conventional 
pedagogy is of little— if any--value in training teachers to 
teach as they were taught -- not in the manner in which they 
learn." 
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“I would like for you to think of a school which would operate 
continuously whenever there are students and where teachers are 
not needed. There would be no faculty in the usual sense. The 
learning systems that the student would use would be based on 
the process of programmed instruction. The purpose of programmed 
learning systems would be to fill a void where there is a serious 
lack of both skilled teaching manpower and effective teaching 
tools for carrying out the needed educational and training programs 
Skill and knowledge requirements in our society are of such magni- 
tude and unique nature that no conventional remedies such as in- 
creasing the number of specialists personnel and material resources 
or building more school rooms and training facilities can satisfy 
our needs in the required time.” 

”What is needed is an inexpensive group of learning systems to meet 
this need directly. What is needed is the engineering development 
of these packaged courses of instruction which will minimize the 
need for competent master teachers and classroom facilities -- sig- 
nificant in those circumstances where competent teachers and ade- 
quate facilities are not available. The programmed instruction 
process is basic to this packaged education." 

"Through programmed instruction we can simulate (or package) the 

tutorial approach. We can lead the student one optimal step at a 

time along the learning path. To one student, it is a small step; 
to another student, it is a large step. We can get the student to 

respond so that h4s response always carries him a little closer to 

the ultimately desired learning." 

"In 1964, Wilbur Schraum, in his review of research in programmed 
instruction, stated unequivocally that five years of intensive 
research demonstrated that students ^ learn from programmed in- 
struction. Programmed instruction, like no other approach to 
teaching that we have had, is dropping essentially "what is dis- 
functional" from learning experience and developing what is truly 
functional." 

"This is not to say that programmed instruction will provide all 
the answers, nor is it to say that it is in and of itself the 
learning technology that our society needs. It does mean, however, 
that its adherents have concerned themselves with tes ting the ex- 
periences vjhich they hypothesize will lead to successful learning. 



"Through programmed instruction we are starting to think about 
applying engineering to solving educational and training problems, 
not to mechanize education, but rather to develop learning tech- 
nology to the point where we can shoot with a bullet Instead of a 
shotgun."... Within ADP there is a tremendous need for training. 
The expansion of the industry itself has necessitated the train- 
ing of thousands of programmers and operators . We are training 
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these people, for the most part, using conventional means. Some 
companies have programmed texts to teach the fundamentals of pro- 
gramming and instruction sets for particular machines. The ef- 
fort to use the ADP system itself to train the people is minimal.” 

"Programmers have time while awaiting debug runSM» Operators have 
long relatively idle periods while long jobs are being run. The 
potential now exists through time share, teleprocessing, and multi- 
processing to use the machine for both production and instruction." 



"The big mistake that is being made is simply mechanizing the class- 
room approach. Some think you merely mechamize the textbook. CAI 
is much more than this . It is the application of the programmed 
learning approach. We need to know specifically what the person 
needs to learn — not just "to learn how to program the XYZ 246." 
^Jhat does a programmer need to know if he is going to be a success- 
ful XYZ 246 programmer? How can we know when he has achieved 
these objectives? What is the best strategy for presenting the 
material to give optimum reinforcement and retention? The tech- 
nician in the ADP feels he can write the course without going 
through all this "Mickey Mouse." However, the student learns in 
spite of, not because of the teaching. It takes time and money to 
do the j job right, but management feels they can shortcut the process. 
Thus, they spend time and money to do the job wrong.” 



"We not only need educators, we need a new breed we might call 
"the educational systems engineer ." We need to experiment and 
extrapolate from our experiments with very little guidanceffrom the 
laboratory of the educational psychologist and learning theorist. 

We need to bridge the gap between basic research and technology. 

We need to study and understand more fully than we presently do 
the impact of the stimulus -configurations (message design) which 
we impose upon 'students and identify those which actually produce 
changes in behavior." 



"Once we do isolate a process or method and say, "Yes, this partic- 
ular set of stimuli presented in this format, under these specific 
conditions to these particular individuals, will reliably produce 
these specific changes in their behavior," the second question we 
must rdise (and now we get to the hardware) is, "How do I instru- 
ment it? How do 1 replicate it? How can I engineer the mass pro- 
duction of the packaged process?" 



"The problem is equally of education in learning design, as well 
as the hardware that we are talking about." 
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APPEiraiX IV 

Selected Organizations Involved 
in Educational Technology 



THE ADI-flLNISTRATIVE HAWAGEMENT SOCIETY 
Willow Grove, Pennsylvania 19090 

A^IERICAN MANAGEMENT ASSOCIATIONS INCORPOPxATED 

135 West 50 Street 

New York, New York 10020 

AMERICAN TELEPHONE AND TELEGRAPH COMPANY 
195 Broadvjay 

New York, New York 10007 

THE Al'IERICAN UNIVERSITY 

Center for Technology and Administration 

Dovaitown Campus 

2000 G Street, N.W. 

Washington, D.C. 20016 

ANALYTICAL ASSOCIATES, INCORPORATED 
420 Lexington Avenue 
New York, New York 10017 

APPLIED DATA RESEARCH, INCORPORATED 

Route 206 Center 

Princeton, New Jersey 08540 

ARRAY INTERNATIONAL DIVISION 
Comspace Corporation 
2^72 Linden Boulevard 
Brooklyn, New York 10108 

THE ASSOCIATION FOR BANK AUDIT, CONTROL, Al-ID OPERATION 

205 West Touhy Avenue 

P.O. Box 500 

Park Ridge, Illinois 

ASSOCIATION FOR COMPUTING MACHINERY 

211 East 43 Street 

New York, New York 10017 

ASSOCIATION FOR EDUCATIONAL DATA SYSTEMS 
1201 16 Street, N.W. 

Washington, D.C. 

AUTOMATION INSTITUTE OF AMERICA 
Subsidiary of C-E-I-R. , Inc. 

760 Market Street 

San Francisco, California 94102 

AUTOMATION SCIENCES, INCORPORATED 

275 Madison Avenue 

New York, Nev; York 10016 
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AUTOitATIOH TRAINING CENTER 
Box 3085 
Papago Station 
Scottsdale 5 , Arizona 85257 

BONNER & MOORE ASSOCIATES, INCORPORATED 
500 Jefferson Building 
Houston, Texas 77002 

BRANDON APPLIED SYSTEMS, INCORPORATED 

30 First 42 Street 

New York, New York 10017 
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C-E-I-R, INCORPORATED 
Institute for Advanced Technology 
5275 River Road 
Washington, D.C. 20016 

COMPUTER COMMAND AI^D CONTROL COI^AiT? 

Suite 1315 

1750 Pennsylvania Avenue, il.W. 

Washington, D.C. 20006 

UNIVERSITY OF CALIFORNIA EXTENSION 
Los Angeles 
California 90024 

CENTER FOR PROGRAI^IED LEARNING FOR BUSINESS 
Bureau of Industrial Relations, Graduate School of 
Business Administration 
The University of Michigan 
340 South State Street 
ihin Arbor, Michigan 48104 



sf 






k 



COLUMBIA Ul^IVERSITY 

Executive Programs, Graduate School of Business 
Uris Hall 

New York, New York 10027 

COMPUTER RESEARCH INSTITUTE 
9506 Magnolia Avenue 
Fdverside, California 92503 

COMPUTER USAGE EDUCATION, INCORPORATED 
Subsidiary of Computer Usage Company 
51 Madison Avenue 
New York, New York 10010 

C0I-2RESS, H'lCORPORATED 
2120 Bladensburg Road, N.E, 

Washington, D.C. 20018 

CONTROL DATA INSTITUTE 



I Division of Control Data Corporation 

I 3255 Hennepin Avenue 

I Minneapolis, Minnesota 55408 

I 
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DATA PROCESSING ^^ANAGEI•^E£•1T ASSOCIATION 
International Administrative Headquarters 
505 Busse Highway 
Park Ridge j Illinois 60068 

DETROIT RESEARCH INSTITUTE 

12 East Hancock 

Detroit , Michigan 48201 

DIGITRONICS CORPORATION 

Albertson 

New York 

DOCUlffiNTATION, INCORPORATED 
4833 Rugby Avenue 
Bethesda, Maryland 20014 

ELECTRONIC C0I4PUTER PROGRAMMLNG INSTITUTE 
Empire State Building 
New York, Nev; York 10001 

EMORY UNIVERSITY 

School of Business Administration 
Atlanta, Georgia 

EMTELEK, INCORPORATED 
Newburyport 
^Massachusetts 01950 

FLORIDA ATLANTIC lElDTERSITY 

College of Business and Public Administration 

Boca Raton, Florida 33432 

FRIDEN, INCORPOPxATED 
Subsidiary of the Singer Company 
2350 Washington Avenue 
San Leandro, California 94577 

GEI^ERAL DYNAl^nCS 
One Rockefeller Plaza 
New York, New York 

GEORGIA INSTITUTE OF TECHNOLOGY 
Rich Computer Center 
School of Information and Science 
Atlanta, Georgia 

INDUSTRIAL MANAGEMEIv^T CENTER 

370 Concord Road 

Weston, Massachusetts 02193 

INDUSTRIAL ^4ANAGE^iENT SOCIETY 
330 South Wells Street 
Chicago, Illinois 60606 

INFORMAiTICS, INCORPORATED 
5430 Van Nuys Boulevard 
Sherman Oaks, California 91401 
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TEE INSTITUTE FOR IIAUAGEIENT AI'ID COIIPUTER EDUCATION, INC. 

135 West 50 Street 

New York, New York 10020 

INTERNATIONAL ACADEUY DIVISION 
LSI Service Corporation 
Washington, D.C. 

INTERImATIONAL ACCOUIJTAi^TS SOCIETY, INCORPORATED 
209 X'Jest Jackson Boulevard 
Chicago , Illinois 60606 

INTERNATIONAL BUSINESS MACHINES 
Education Department 
Poughkeepsie, New York 

niTERNATIONAL CORRESPONDENCE SCHOOLS 
Scranton 

Pennsylvania 18515 

LEART^ING FOIEJDATION INSTITUTE 
Executive Division 
505 Fifth Avenue 
New York, New York 10017 

LING-TEMCO-VOUGHT INCORPORATED 

Arlington 

Texas 

MAJJAGEl'ffiNT DEVELOPlffiNT INSTITUTE 
130 West Lancaster Avenue 
Wayne, Pennsylvania 19087 

MANAGEMENT S9QX39|jaiMWINING INSTITUTE 
A Division of the Dlebold Group, Inc. 

430 Park Avenue 

New York, New York 10022 

Ui^IVERSITY OF MCHIGAtJ 

Institute of Science and Technology 

College of Engineering and Extension Services 

Conference Department 

412 Maynard Street 

Ann Arbor, Michigan 48104 

3M COI'IPANY 

Visual Products Division 
Box 3100 

St. Paul, Minnesota 55101 

NATIONAL ASSOCIATION OF ACCOUNTAI'ITS 

505 Park Avenue 

New York, New York 10022 

NATIONAL BUSINESS SYSTEM ASSOCIATION, INCORPORATED 
5856 Northwest Highway 
Chicago, Illinois 60631 
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NATIOML CASH REGISTER COI4PANY 
Main & K Streets 
Dayton, Ohio 45403 

NATIONAL MICROFIIM ASSOCIATION 
R. 0. Box 386 
250 Prince George Street 
Anni5q»61is , Maryland 21404 

UNIVERSITY OF NEW MEXICO 
Albuquerque 
I New Mexico 87106 

I NSW YORK UNIVERSITY 

I The Management Institute 

School of Continuing Education and Extension Services 
I Washington Square 

I New York, New York 10003 



I PHILCO-F^ CORPORATION 

I TechRep Division 

1 Tioga and C Streets 

$ Philadelphia, Pennsylvania 19134 

I 

I PURDUE UNIVEI^ITY 

I Lafayette 

I Indiana 47907 
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PROGRAMMING AND SYSTEMS INSTITUTE 

33 West 42 Street 

New York, New York 10036 

THE SERVICE BUREAU CORPORATION 
1350 Avenue of Americas 
New York, New York 10019 

SCIENTIFIC METHODS, INCORPORATED 
Box 195 

Austin, Texas 78767 

SOUTHERN ILLINOIS UNIVERSITY 
Computer Center 
Department of Management 
Vocational Technical Institute 
Carbondale, Illinois 

STANFORD UNIVERSITY 

Standford 

California 

SYSTEMATION, INCORPORATED 
P. 0. Box 1188 
Tulsa, Oklahoma 74101 

SYSTEM DEVELOPMENT CORPORATION 
1923 Centinela Avenue 
Santa Monica, California 
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SYSTEMS AND PROCEDURES ASSOCIATION 
P. 0. Box 549 
Contrecoeur 
Quebec, Canada 

UNIVAC DIVISION 
Sperry Rand Corporation 
1290 Avenue of the Americas 
New York, New York 10019 

THE UNIVERSITY OF WISCONSIN 
University Extension 
432 North Lake Street 
Madison, Wisconsin 53706 

WHARTON SCHOOL OF FINANCE AND COMMERCE 
University of Pennsylvania 
Department of Industry 
Philadelphia, Pennsylvania 19104 
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MAJOR CAI CENTERS 



Dedicated Systems 

IBM Yorktotm 
IBM Poughkeepsie 
IBM Los Gatos 

University of Califomia/irvine 
University of California/Santa Barbara 
Darmouth College 
MIT MAC 

Systems Development Corporation 
AF Electronic Systems Div. 

Bolt, Beranek and Newman 
Penn State University 
University of Illinois 
Harvard Computation Laboratory 
Florida State University 
University of Texas 
University of Pittsburgh 
Stanford 
Westinghouse 

Public Utilities 



General Electric Time-Sharing Service 
Bolt, Beranhk and Newman /TELCOMP 



